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ABSTRACT 



An apparatus, functioning as a user-input device (UID) to a 
computer by using a communication link, receives/transmits 
additional I/O signals via the same link. The I/O signals may 
be analog or digital. One implementation of the apparatus, 
made wireless and supported by speech processing software, 
allows its user to interact with the computer remotely. The 
inventive apparatus comprises means for converting the I/O 
signals into/from the digital I/O streams in such a manner as 
to be transferable to/from the computer along with the UI 
information via the same communication link. The 
apparatus, while functioning as a UID, may handle the I/O 
signals coming from/to an external I/O device, such as a 
fax/modem device or even another conventional UID. The 
apparatus, while performing as a UID, may include the 
complete functionality of an external I/O device handling 
the I/O signals. A communication program method, by 
which a computer receives a combined data stream resulted 
from two sources, one from a UID another from an addi- 
Uonal input signal, recovers from such stream the respective 
information of the two sources. 

25 Claims, 8 Drawing Sheets 
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METHOD AND APPARATUS FOR 

PRODUCING A COMBINED DATA STREAM J^J^^to handles the I/O 'speech signals 

AND RECOVERING THEREFROW [THE "J^S^&n with a sound card in order to deal 

RESPECTIVE USER INPUT ^TTOAM ^AND speech inputs and two speech outputs. Such use is 

AT LEAST ONE ADDITIONAL INPUT 5 £ computers that need to deal with two sources of 

SIGNAL s ch inpiltSj My> 0D e as the voiced command and another 

for multimedia recording. Furthermore, the inventive appa- 
FffiLD OF THE INVENTION [^^Thllles the speech I/O signals will be further 
This invention relates to computer user-input devices supp orted by software ^^^^^ " 
including pointing devices especially to those handling ^^sSWiSSL^ 
additional input/output signals. ^ZT^cro^JL * speaker, will allow the user 
BACKGROUND OF THE INVENTION to interact with the computer remotely. By carrying the 
num* ™ w apparatus with her, the user may issue oral commands to the 
A computer utilizes a user-input device (UIU) to receive lS « ^ ^ microphone) and hear its responses from 
its user's input and various kinds of input/output (I/O) cards ^ speakeij while being away from the computer, 
or devices to handle I/O signals or information. A typical Q ^ Qb ^ c{ q[ ^ inventlon pro vides a user-input (U0 
UID is the mouse, the trackball, the keyboard, the pressure atus mat scn d s its data to a computer, by using the 
tablet, or the pen-based input device. Popular I/O cards commU nicatioa link available to a conventional UID, while 
include the sound card handling I/O speech signals and the 2Q reccivi ^ transm i u ing additional I/O signals, the informa- 
f ax/modem device transferring information over the tele- ^ of which being transferred to/from the computer via the 
phone line. Devices and cards require and share common communication link. The inventive apparatus corn- 
computer resources such as the direct memory access g m ^ for convertmg the I/O signals into/from the 
(DMA) channels and the interrupt request (IRQ) services. ^ yQ streams m sucn a man ner as to be transferable 
Computer resources for each device or each card equipped 25 me computer along with the UI information via the 
in a computer must be configured, or assigned, to pre- same commun i ca tion u^. The apparatus, wrule tunctiomng 
arranged memory locations that are limited in number. as & may handle I/0 signals sent from/to an external 
Configuration setup for computer resources is cumbersome {/Q &uch as m extema i fax/modem device or even 
and sometimes causes conflict in running software. The conventional UID. Furthermore, there is a program 
recent plug-and-play technology implemented in computers 30 memod ^ execute d by the computer and cooperating with the 
alleviates the configuration mechanism. However, it does apparatuSj f 0T transferring information over the communi- 
not solve the problem of limited resources shared among cation ^ and for reco vering therefrom the UI information 
cards and devices. As computer technology advances, more ^ ^ j^g^^ 0 f the input signals, 
types of cards and devices arc offered for richer sets of Two examples of communication protocols used by a UID 
functions; efficient use of computer resources becomes 35 ^ m opular RS _232 and the newly standardized Univer- 
critical. Furthermore, a conventional computer requires that ^ ^ ^ ^g^ion 0 f the communica- 
its user be close to its display screen in order to see and ^ & may be (yia or ^less (via 
control what shown on the screen. If somehow the user clectromagnetic wavc ). The I/O signals may be analog, such 
could issue commands to her computer and receive its as the speecD? or digital, such as the data transferred to/from 
responses remotely, she would gain a new freedom, such as 4Q ^ ^ 

walking around or staying in bed, while using the computer. fa ^ ^ ^ & UID> ^ inventive apparat us 

The present invention offers a new kind of UID utilizing tne ^ ^ mQ(mle as one of its components or 

computer resources efficiently and enabling a mode 01 , ti d prov ides interface to an external conventional 

remote interaction between the computer and its user. ^ * * 

SUMMARY OF THE INVENTION 45 Another object of the invention provides an apparatus 

bUMMAKi w in* comprising only the feature of receiving at least one input 

An object of the present invention provides an apparatus x 

functioning as a UID to a computer while receiving/ ^ inventiyc apparatus may integrate means to handle 

transmitting additional I/O signals transferred to/trom tne ^ ^ ^ ^ ^ ^ microphone converting a speech 

computer without using any computer resources except 50 mput mto ^ input signa l and/or the speaker converting the 

those available to the UID. output signal int0 speecn energy. 

Since the inventive apparatus is a UID functioning addi- ^ invenlive apparatus may be implemented, at least 

tionally as an I/O card (or device) without using the card (or integrated circuit (IC) technology using such 

device) and its resources, it provides the following advan- v ^ ^ ammable i ogic device (PLD) or the 

tages. Elimination of the I/O card (or device) reduces the 55 J fofi ^ IC (ASIC). 

computer parts and resources, thus enhances the computer s * ^ may be imp iemented, at least 

reliability, serviceability and efficiency. Such inventive ^ microprocessor-based design; furthermore, 

apparatus is useful in hand-held computers or embedded P compression/decompression algorithms may be used to 

systems, which usually have a communication port needed ^ ^ ^ ^ rommudcauorj 1^. 

for a UID but no provision for I/O cards or devrces) Jhen 60 be d ^ ffom the 

handling two input (or output) signals of th signals of the communication link. 

identical I/O cards cannot be used in one computer due to invention provides a method for 

time as a UID. A computer may use such apparatus in signal. 
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nDicB nPsrniPTION OF THE DRAWINGS One embodiment of the present invenuon, having the 

BRIEF DESCRIPTION OF iHfc ukawihu* fmi6oBA b i ock diagram depicted by FIG. 2A, comprises a 

In the figures referred below, the like numbers or names yj means 20 ( a signal converting means called converter 30, 
denote the like elements. a framing logic means called framer 34, and a commumca- 

FIG. lAshows a computer utilizing a conventional UID s 
and a sound card. ^ bbck ^ ^ shown FIG. 2C in 

FIG. IB shows the same computer wherein one embodi- ™™ ^ ^ ^ ag ^ fax/modem device , 

rnent of the invention replaces both the UID and roe sound teaffimi ^ receives 1/0 signals to/from converter 30 via input 
card. 32 and output 53. In reference to FIGS. 2A, 2B, and 2C, UI 

FIGS. 2A, 2B, and 2C show block diagrams of the various 10 mean& 20 produces tji stream 24, while converter 30 con- 
embodiments of the invention. verts input signal 31, delivered by input 32, into input stream 
FIG 3Ashows one embodiment of the invention utuizing 33 representative of the input signal. Framer 34 keeps UI 
the RS-232 cable stream 24 and input stream 33 in synchrony and encodes 

it . • f f thP H<5 w data them into combined data stream 37 in accordance with the 

FIG. 3B shows one timing waveform of the RS-232 data ^ communication means. FIG. 2C additionally 

format. shows converter 30 receiving output stream 54 from framer 

FIG. 4A shows one logic design of another embodiment ^ ^ prQduoe outpm 52 go ing out via output 53 to the 
of the invention. external device. Generally, any stream described in this 

FIG 4B shows the timing waveform of the critical signals invention has digital values, which may be transported either 
involved in FIG. 4A. 20 by a single signal or by multiple signals. Communication 

FIG 5 shows the block diagram of one of the onboard me ans 35 translate combined data stream 37 into .proper 
em bodimen,ofmeinven,ion. ^^^SS^^^ 
DESCRIPTION OF THE PREFERRED drawings, the external device, link36, computer 10, and port 

EMBODIMENTS 25 38 do not belong to the embodiment of the invenuon. For the 

• um ™ m „„t„r I/O sienals beine analog, converter 30 at least performs as 
FIG. lAillustrates a computer system in which computer W*g»o« converter such as the encoder-decoder 
lOcommumcateswithUIDllviacommumcaUonlmkUto S^taXw, signals being digital, the converter at 
move object 14 displayed on monitor 13, whrle uUhzing (codec). For «*WJJ .JV&amer m ay directly 
sound card 15 to revive speech input &om microphone 16 "jj"" ^erge jigM N»» signal 31 with UI 
and to transmit speech output to speaker «. Accontag to 30 ^™ ' K ^ strelm 37 . 

oneimplementationofthepre^nt^ tier to taplement wireless communicaUon link, com- 

U1D 11 with inventive apparatus 18, as depicted m Flu. IB in oraer uiimpie m transceiver which 

and deserved f^SZ^T^ Srrthe^l^sS forming link 
sound card. Apparatus 18 directly receives speecn upu compatible wireless trans- 

fr ommicrophonel6and^,«sspeecho„,puttos P e^ 35 ^ e ^£~ the ^J^. Two typical 
17. When using the computer shown m FIG. IB, ^ a user cMverto ™^ infrared-light (IR), such as 

operates apparatus 18 normally as a conventional UID and ^e*^™ ™ ^ and & t ?J b „ 

s frith""- 



U1D 115.2 kbits/s. 
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In applications where the efficiency of the communication does not need transceiver 35; framer 34 hriks tody t •the 
JkkS^fSl in wireleJcommnnication, com- universal asynchronous recerver/transmUter (UART) con- 
bined data stream 37 is encoded into a single signal trans- troller 39 existed inside computer 10. 
mitted by a single wire or electromagnetic wave. FIG. 4A Working in conjunction with each inventive apparatus, 
depicts one embodiment of the invention in which framer 34 s there exists a program method, executed by the computer, 
encodes data stream 37 into a single signal. In reference to for sending output information to the apparatus, for receiv- 
FIG 4A, the codec TP3054, in place of converter 30, tDe combined data stream 37 in which a UI stream and 
converts output information 54, sent from the computer via M ^ stre am are kept in synchrony and encoded and for 
line TXD, into output signal 52 entering output 53. recovering therefrom the respective information of the Ul 
Concurrently, the codec also converts input signal 31, deliv- io str eam and of the input signal. The program method corn- 
ered by input 32, into serial stream 33. In order to make prises m initialization section, an interrupt service routine 
stream 33 compatible with the RS-232's data format, a (1SR)) and a data recovering routine. Specifically, for the 
start-bit pulse provided by signal STA* is logically AND-ed inventive apparatus communicating with a personal com- 
with stream 33 to become digital input stream SX carrying puter (pq via an RS-232 link, the three parts of the method 
information of the input signal Since codec 30 converts 15 are a s follow. 

output information 54 and input signal 31 simultaneously, ^ initialization section comprises the steps of: 

their corresponding data streams traveling on the RS-232's essential data buffers and their respective 

data lines, TXD and RXD, are preferably synchronized for * £ 

their timely transmission and reception by the computer. [M '^ C communication control signals; 
One logic implementation of the framer, as depicted in FIG. 20 initializing mc «j 

4A, relies on the signal of line TXD to maintain the programming the communication mode, 

synchronization between the UI and input streams. The data installing the interrupt service routine; 

lines TXD and RXD are programmed by the computer to loading initial data mto the transmit-data register; and 

have a data rate of 115.2 kbits/s and a frame format enabling the communication interrupt to transmit/receive 

* composed of eleven bits: one start-bit, eight data-bits, one 2 5 data 

parity-error-bit (PE-bit) and one stop-bit. In reference to ^ interrupt service routine comprises a tightly written 

FIGS. 4A and 4B, flip-flop FF2, utilizes signal CLR to ^ for; 

re-synchronize clock generator 62 with the start-bit of signal ^ ^ rammunicatkm sta tus; 

Z TtX high voltage VCC at FF2's input pin, it waits for transmitting data from a transmit-data buffer, 

bemg set h{h by the upcoming start-bit of signal TXD*. monitoring to receive and store data in a receive-data 

Once set signal CLR restarts, and is immediately reset by buffer; 

the same, clock generator 62 providing bit-clock signal deriving the Ul-bit associated with eachreceive-data byte; 

BCLK of the codec. Signal BCLK is divided by sixteen to 35 ^ 

become signal QH, which is latched into flip-flop FFI to stQring me iji-bit into an Ul-bit buffer. 

provide the codec's frame-sync signal FS converting input Depending on the way the combined data stream 37 is 

signal 31 into serial stream 33 coming out of the codec's pin forme ^ me step of deriving the Ul-bit is done as follows. If 

DX Simultaneously, signal FS moves output information 54 stream 3? is sent via tw0 sepa rate signals, as shown in FIG. 

serially into the codec, via pin DR, to be converted into 40 3A> thal step reads the level of line CTS representing the 

output signal 52. Both UI stream 24 and input stream SX are y ^ of the m_bit during each receive-data byte. If stream 

synchronezed and encoded into a single signal RXD* by 3? fc gent via one single signal, as shown in FIG. 4A, that 

data selector 50 under control of signal SEL. The latter, stepr eads the parity error status, compares with the expected 

formed by signals coming from clock generator 62, pro- pE _ bU of lhe rece ive-data byte, and derives the correspond- 

duces a string of positive pulses in sychrony with the PE-bit 45 • UT _ bit( p or example, corresponding to a receive-data 

locations of signal RXD*, referred to as the Ul-bit and byt e = 5D-hex (expected PE-bit is even), the Ul-bit is 0 if an 

denoted as "UI" in FIG. 4B. Data selector 50, normally odd . parity error 0CC urs otherwise 1. 

selecting the input stream SX, otherwise samples UI stream ^ data recove ring routine has the steps of: 

24 and stores its data into the Ul-bit location of signal covering the input-signal byte from the data contained 

RXD*. Since UI stream 24 varies slowly compared to the 50 ^ ^ receiv e_ d ata buffer; 

pulsing of signal SEL, its information is conserved in, and input-signal buffer; 

subsequently can be recovered from the U -bits 0 signal P ^ each byte of the UI 

RXD* The Ul-bit location on signal RXD* is not neces- reconstructing now 

sarily at the PE-bit. For instance, the least-significant data- stream, ana 

hit location mav be used instead. In such case, the 55 depositing it into the Ul-butrer. 

™lis of Sal SEL occur at the DO-bit time slots; The reconstruction of each Ul-byte takes into account the 

entirelv from sknal RXD* to enhance the data transfer rate. each data frame associated with that Ul-byte. tor example 

To £ to ft signals, oscillator OSC of in reference to FIG. 4B, the ^UART » programmed wiOi 

frecSiencv JZ lS MHz generates master clock MCLK 6 o eleven bits at 115.2 kbits/s white the UI stream received at 

S^^p^O^^UOX 1200 bits*; ^ 

hv sixteen to croduce bit clock BCLK. The timing waveform corresponding to 11 5. 2*<1. 2*ll)=8/72 received data oytes^ 

for sknak SEU FS QH R*, TXD*, and RXD* are Therefore, each Ul-bit must have the same bit-value for N 

d3dt TO Sn relato to bit clock BCLK. consecutive receive-data bytes, where N * an integer near 
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start-bit, seven data-bits representing the Ul-byte, and one encoding means for synchronizing the user input stream 

stop-bit. Due to the asynchronous nature of the UI stream, with the input stream and encoding the same into a 

the stop-bit may have the bit length N>8.72 before the combined data stream transferable by the ccmmunica- 

occurrence of the start-bit of the next Ul-byte. tion means. 

For the apparatus that receives only input signals to be s 2. The apparatus of claim 1 wherein the combined data 

sent to the computer, the UART does not transmit data. stream j s encoded into a single signal. 

Accordingly, the first two steps of the program method are 3, The apparatus of claim 1 wherein at least part of the 

modified so 'that the ISR will be invoked when its receive- apparatus is implemented by an integrated circuit device, 

data being available, rather than when its transmit-buffer is 4 jhe apparatus of claim 1 wherein at least part of the 

em pty. 10 apparatus is implemented by a microprocessor. 

For the inventive apparatus connected to a computer by ao 5 The apparatus of claim 4 further comprising means for 

RS-232 cable, its power can be solely drawn from the compressing/decompressing the combined data stream, 

RS-232's output signals, such as the RTS, DTR, and TXD; reducing the data traffic traveling on the communication 

hence, the apparatus needs no battery or external power means. 

supply. 15 6. The apparatus of claim 1 wherein the apparatus resides 

Also, the 1C technology, using the PLDs, the gate arrays, m me computer, 

the ASICs, or the mixed-signal ICs, can integrate many 7. The apparatus of claim 1 wherein the apparatus draws 

elements of the apparatus into at least one IC device. For its power solely from communication signals traveling on 

instance, a combination of the framer, the codec, the elec- mc communication means. 

tronic part of the UI means, and the transceiver, can be ^ g. The apparatus of claim 1 wherein the communication 

packaged into at least one IC device. Alternatively, the logic mcans ^ an RS-232 having at least two separate signal lines 

of such combination can be implemented by a wherefrom the encoding means sends the user input stream 

microprocessor, such as part DSP56xxx from Motorola. and the input stream. 

The above-mentioned TP3054 codec, compresses/ 9, The apparatus of claim 1 wherein the communication 

decompresses the I/O signals into data according to the 25 means ^ a Universal Serial Bus. 

simple mu-law; there are other known methods that do so 10. The apparatus of claim 1 wherein the input means is 

more efficiently, such as the ADPCM in the communication an mput transducer. 

field and the vector-quantization (VQ) in the speech pro- u The apparatus of claim 1 wherein the user input means 

cessing field. Such methods can be implemented by a ^ m interface to an external user input device, 
microprocessor to reduce significantly data traffic on the 30 12. The apparatus of claim 1 further comprises means for 

communication link. receiving an output stream from the computer via the 

The above implementation converts straightforwardly the communication means and means for converting the output 

analog I/O signals into/from the data streams sent to/from stream into at least one output signal, 

the computer. Other embodiments of the invention have the 13 The apparatus of claim 12 further comprises an output 
converter process the I/O signals completely before trans- 35 transducer converting the at least one output signal into 

ferring their information to/from the computer along with outpilt energy. 

the UI information, such as the case of dealing with the 14 A programming method, executed by a computer 

telephone I/O signals. communicatively coupled via a communication link to a 

For the apparatus using the USB as the communication user mput mea ns having means for synchronizing and 
link, a USB controller, such as part CY7C6xxxx from 4Q encoding a user input stream and at least one additional input 

Cypress Semiconductor, 3901 North First St., San Jose, s i gna l into a combined data stream, comprising the steps of : 

Calif. 95134-1599, implements either only transceiver 35 or initializing the communication link; 

both framer 34 and communication means 35. More serv icing a single resource service interrupt for receiving 

generally, a micro-controller or a microprocessor specialized ^ combined dala stream; and 

in the USB protocol can be used to implement at least some 45 ^ ^ ^ combiaed daU stream respective 

electronic parts of the inventive apparatus. information of the user input means and of the at least 

Another aspect of the invention offers a communication ^ ^ 
method by which a computer, receiving a combined data ^ mmi method of claim 14 further corn- 
stream associated with two sources of information, one from . ^» yia me communication link, output 
a UI means and another from an input signal, recovers from 50 V information being received and 
such data the respective information of the two sources. resi ding in the user input means 
Such method comprises the three steps described earlier. 

Such method further transmits data onto the communication. apparatus linked to a computer by a communica- 

Although the present invention has been described in ^ jf^^ B a user input devicc and additionally 

connection with the above-mentioned embodiments £ is not 5S comprising: 

limited to their illustrated implementations. People stalled in J* * ^ ^ 

the art can modify, enhance, and/or refine those implemen- a user mpui aeviu; pi B y 

rations to achieve the spirit and cover the scope of the an input port receiving at least one mput signal, 

invention as defined in the appended claims. a converter receiving the at least one input signal tor 

I c iaim: 60 producing an input stream; and 

1. A user input apparatus operatively coupled to a com- a framer synchronizing the user input stream with the 

puter via a communication means additionally receiving at mpu t stream and encoding the same into a combined 

least one input signal, comprising: data stream transferable by the communication link, 

user input means for producing a user input stream; 17. The apparatus of claim 16 further comprises means for 

input means for producing the at least one input signal; 65 receiving an output stream from the computer via the 

converting means for receiving the at least one input communication link and means for converting the output 

signal and producing therefrom an input stream; and stream into at least one output signal. 
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18. THe apparatus of claim 16 wherein the converter a user input device for producing a user input stream; 
further comprises an output port wherein the framer further aQ inpm port for pro ducing the at least one digital input 
receives an output stream from the computer via the com- . 

munication link, the output stream being further received » 

and converted by the converter into at least one output signal s a f ram6r for keeping the user input stream and the at least 

going to the output port. one digital input signal in synchrony and encoding the 

19. A method by which a computer receives via a com- game [n{Q a ^^1^ data stream transferable to the 
munication link a combined data stream containing synchro- communication link. 

nized and encoded information of a user input means and of compuier D y inc cuww 

aOeast one additional input signal, said method comprising 22. The apparatus of claim 21 wherein the framer further 
the steps of: 10 receives output information from the computer to provide at 

initializing the communication link; least one output signal. 

servicing a single resource service interrupt for receiving ^ ^ a p paratus 0 f claim 22 further comprises an output 

the combined data stream; and transducer converting the at least one output signal into 

recovering from foe combined data stream respective 

information of the user input means and of the at least £ ^ ^ ^ ^ ^ ^ apparatus is 

20°£e a l^ofSWfiutte comprises transmit- fiirther coupled to an external device, the external device 

ting output information to the communication link, the providing the at least one digital input signal and receiving 

output information being received and converted by a con- ^ at i east one output signal. 

verter residing in the user input means into at least one 2 o 2 5. The apparatus of claim 21 wherein the input port 

output signal. receives the at least one digital input signal from an externa 

21. A user input apparatus operatrvely coupled to a ™ » 

computer via a communication link receiving user input device. 

signals and additionally at least one digital input signal, ***** 
comprising: 
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